A Concept Work for Augmented Reality Visualisation Based on a Medical Application in Liver Surgery

This paper describes a concept of an innovative medical application with augmented-reality support. This work is based on a cooperation between the Div. Medical and Biological Informatics at Deutsches Krebsforschungszentrum (DKFZ), and the Department of Photogrammetry and Cartography of the Technical University Berlin. Augmented Reality (AR) is a growing area in virtual reality research. An augmented reality system generates a composite view for the user. It is a combination of the real scene viewed by the user and a virtual scene generated by the computer that augments the scene with additional information. The wide scope of application domains reveals that the augmentation can assume a number of different forms. Therefore, we compare different technologies for augmented reality visualisation with the focus on video seethrough head-mounted displays, optical see-through head-mounted displays and virtual retinal displays. In a further step, we analyse the state-of-the-art of medical applications in augmented reality visualisation. New technologies and methods implemented in DKFZ’s ARIONTM software will enable image-guided liver surgery. It is shown that many tools for preoperative planning of surgical interventions in hepatobiliary and pancreatic (HBP) surgery are available in the clinical environment, while the surgical procedures itself still lack computer assistance. On this basis, Div. Medical and Biological Informatics at DKFZ, the Depts. of Surgery and Radiology at the University Clinics Heidelberg and the research group “AR-work” of Technical University Berlin derived strategies for AR-support of open liver surgery. A technique will be developed in order to superimpose virtual computer-generated information with real patient image data using a see-through technology. The data from the preoperative planning, the clinical and technical requirements for augmented-reality techniques and our concept of the augmentedreality system will be presented in this contribution. The advantages of see-trough technology will be discussed. Our future work will analyse why (nowadays) AR-systems are not available in surgery and which technical and clinical requirements can be fulfilled with currently available display technologies.